AI Lessons
Lesson 22: Tokens and Embeddings - How AI understands

Purpose: To understand how AI converts data into tokens and the meaning of data into embeddings.

No. of Classes

1 - (Time : 1 hour 30 minutes, Laptops/desktops : 10, Students strength : 15 to 20).

Materials Required

Laptop / Desktop with Internet connections / Wi-Fi/ Tokens and Embeddings activity worksheet.

Curriculum Content

2. AI and ML Concepts - A Simplified Overview

  • 2.8 Tokens
  • 2.9 Embeddings
Prior knowledge
  • Basic understanding of AI and text processing
  • Concept of how computers process information by identifying patterns in data
Exercises

Exercise (1)


  • Count the words in different sentences and explore how AI breaks text into smaller units called tokens. Learn that AI also converts images, audio and videos into tokens for processing.

Exercise (2)


  • Compare the meanings of different sentences and group similar ones together. Discuss how AI uses embeddings to represent similar meanings closer together and different meanings farther apart.

Solutions



Teacher's Instruction:
  1. Emphasize the words tokens and embeddings.
  2. Tell the students that words or sentences with similar meanings are placed close together.
  3. Explain that words or sentences with different meanings are placed far apart.
  4. Points to Ponder:
    • We saw the concept of embeddings in words and sentences. How do you think other types of data are represented as embeddings or are embeddings necessary for other types of data? (e.g., images, audio, video, etc.)